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ABSTRACT 

This  report  is  a  continuation  of  earlier  studies  in  the  development 
of  the  theory  of  differential  games  with  information  time  tag  (DGWITL) . 

Pertinent  aspects  of  the  theory  of  differential  games  with  information 
time  lag  are  summarized.  The  generalized  Hamilton-Jacobi  equation  for  the 
linear  regulator  game  is  shown  to  have  a  o-parametric  solution  in  the 
data- reference  plane.  The  main  effort  in  this  report  has  been  to  provide 
examples  which  demonstrate  both  the  applicability  of  the  theory  of  DGWITL 
and  the  importance  of  information  delay  considerations  in  control  law  synthesis. 
Examples  illustrate  the  performance  degradation  that  can  accompany  a  failure 
to  compensate  for  information  delay  in  differential  games.  At  the  same  time, 
the  quantitative  effects  displayed  in  the  examples  serve  to  motivate  questions 
relevant  to  engineering  tradeoffs  between  control  law  design  and  system  design. 
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I.  INTRODUCTION 


In  discussing  some  possible  extensions  of  his  work  on  differential  games  (DG) , 
Isaacs  mentioned  the  need  for  a  relaxation  of  the  assumption  that  each  player  in 
a  DG  has  perfect  information  [11.  In  many  practical  situations  of  dynamic  conflict, 
the  assumption  cannot  be  justified.  A  particular  problem  which  Isaacs  cited  is 
that  in  which  a  player  it  a  DG  has  a  time  lag  associated  wltn  the  availability 
of  his  measurements  of  the  opponent’s  state  vector.  In  practice,  such  delays  can 
occur  in  a  DG  when  data  processing  and/or  transmission  takes  place  prior  to  the 
generation  of  control  signals.  The  importance  of  the  problem  stems  from  the 
practical  need  to  know  precisely  how  much  Information  delay  leads  to  Intolerable 
performance  degradation.  At  the  same  time,  knowledge  of  the  effects  of  time  delay 
might  permit  relaxation  of  system  design  specifications  to  achieve  an  overall 
economy  of  design  while  maintaining  acceptable  performance. 

The  historical  basis  for  the  problem  appears  to  be  the  classical  "bomber- 
battleship  duel",  in  which  a  bomber  must  take  into  account  ordnance  delivery  time 
in  deciding  when  to  strike  an  evasive  battleship.  Since  1950,  a  number  of  papers 
have  successfully  treaced  the  bomber-battleship  duel  as  a  multistage  game  with 
probabilistic  strategies  [2-9],  However,  no  treatment  of  the  more  general  problem 
was  presented,  and  little  was  known  about  the  possible  affects  of  information 
delay  in  dynamic  conflict.  Subsequently,  Ciletti  [10]  treated  finite  games, 
multistage  games  and  a  class  of  differential  games  in  which  one  player  has  an 
information  time  lag.  Function  space  methods  provided  the  solution  to  versions 
of  the  well  known  linear  regulator  differential  game  with  information  time  lag 
(DGWITL)  in  [11,  12,  13].  The  above- referenced  works  showed  that  a)  the  player 
with  information  lag  can  construct  a  payoff  bound  and  an  associated  worst-case 
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strategy  by  considering  past  as  well  as  future  behavior  of  his  opponent, 
b)  an  optimal  control  problem  must  be  solved  to  provide  the  boundary  condition 
for  the  game  with  information  time  lag,  and  c)  the  opponent's  strategy  which 
Induces  the  worst-case  payoff  -  an  ideal  exploitation  strategy  -  is  not 
physically  realizable  because  it  requires  knowledge  of  future  states.  These 
structural  features  were  shown  to  be  common  to  finite,  multistage  and  differential 
games  with  information  time  lag.  N-player  DGWITL  (linear  plants,  quadratic 
payoffs)  also  admit  to  analysis  by  function  space  methods  [14].  Despite  the 
availability  of  results  for  the  linear  problem,  there  was  a  lack  of  results  for 
more  general  systems  and  payoffs.  Then,  an  extension  of  the  well-known  Hamilton- 
Jacobi  theory  fox  optimal  control  and  the  familiar  "main  equation"  analysis  of 
Isaacs  was  developed  to  treat  DGWITL  [15].  The  Hamilton-Jacobi  equation  was 
shown  to  have  a  dual-time-reference  form  in  which  the  observation  times  (r,t)  of 
the  state  vectors  (x(r),y(t))  as  well  as  the  spatial  data  appeared.  It  was  pointed 
out  that,  In  general,  an  optimal  control  problem  must  be  solved  to  provide  the 
boundary  condition  for  the  potential  value  function,  V°(x,T,y,t)  over  the  x-t 
data  reference  plane.  The  generalized  Hamilton-Jacobi  equation  for  V°(x,T,y,t) 
over  the  i-t  plane  can  be  converted  to  a  partial  differential  equation  in  t 
along  the  line  x  *  t-a  for  fixed  a  to  produce  the  equation  satisfied  by  the 
parametric  solution,  V°(x,y,t;a)  [16],  In  an  independent  work,  Sokolov  and 
Chernous'ko  [17,  18]  showed  that,  under  certain  conditions  of  separability,  a 
class  of  DGWITL  can  be  considered  equivalent  to  a  related  DG  without  information 
time  lag.  Petrosjan  [19]  treated  a  discretization  of  a  DGWITL  within  the  context 
of  mixed  strategies. 

This  paper  continues  the  developments  of  [15,16].  Section  II  briefly  suana- 
rizes  the  main  results  of  [15,16],  and  Section  III  demonstrates  the  application 
of  the  theory  for  DGWITL  to  the  well-known  linear  regulator  game.  The  problem  will 
be  solved  in  closed  form  over  the  data  reference  plane  to  provide  feedback  control 
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laws  which  optimally  utilize  the  delayed  information.  We  also  present  the 
specialized  solution  along  the  locus  of  play  for  fixed  o.  Structural  properties 
which  distinguish  DGUITL  control  laws  from  suboptimal  control  laws  are  examined. 
Section  IV  presents  results  of  practical  significance.  By  means  of  a  simple 
two-dimensional  example,  wo  examine  the  precise  effects  of  information  time  lag 
on  the  outcome  of  a  game.  Then  we  consider  alternate  control  schemes  which  may 
be  suggested  for  use  in  DGWITL.  Simulation  results  show  the  effect  of  using  the 
DGWITL  optimal  strategy,  a  direct  insertion  feedback  strategy  (DIFBK) ,  and  a 
simple  predictor  feedback  scheme  (PRFBK) . 

II.  SUMMARY  OF  DGWITL  THEORY 

As  a  point  of  reference  for  the  ensuing  work,  we  begin  with  the  dynamic 
systems  controlled  respectively  by  the  pursuer  (P)  and  the  evader  (E) : 

x  =  f(x,u(t),t)  (la) 

y  =  g(y,v(t),t)  (lb) 

*\j  q  th  % 

when  x,y  e  R  ,u  e  R  ,v  e R  •  At  each  time  t  the  control  inputs  u  and  v  are  to  be 

determined  by  control  laws  u  and  v  which  are  members  of  sets  and  K^,  where 

K  =  {u:Rn  x  Rn  x  R1  -*■  A  C  R<1}»  K  =  <v:Rn  x  Rn  x  R1  -»•  A  CR*},  A  and  A  are 

u  u  V  vw,uv 

locally  compact,  an4  -’embers  of  Ku  and  are  continuous  in  t  and  Lipschitzian 

in  (x,y).  Given  regions  G^  and  Gv  to  which  motions  of  (la)  and  (lb)  are  restricted 

and  a  target  set  SCG  =  G^  x  Gv,  we  form  C  Ky  x  to  include  only  those  control 

law  pairs  which  produce  for  any  initial  phase  in  G-S  a  solution  which  reaches  S 

without  leaving  G.  Accordingly,  the  terminal  time,  tf  (depending  on  xQ,yo, t?, u,v) 

is  the  first  instant  at  which  a  solution  to  (1)  penetrates  S  .  With  (u,v)  e  K  , 

A 

(x  .t  ,y  ,t  )  e  G,  we  associate  a  scalar  payoff  made  by  P  to  E  and  defined  by: 
o’  0  0  o 
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J(x  ,y  ,t  u,v)  *  I  L. (x(a) ,u(x(a) ,y(a) ,a) ,a)da 

O  O  O  Cl 

o 

Cf 

+  t /  L2(y(a) ,v(x(a) ,y(a) ,a) ,a)da 

o 

+  W(x(tf),y(tf),tf) 

where  x(»)  and  y(»)  are  understood  to  be  the  solution  to  (1)  associated  with 
the  control  law  pair  (u,v). 

The  methodology  of  game  theory  then  centers  about  the  task  of  finding  a 
saddlepoint  of  J(*)  w.r.t.  u  and  v.  The  well-known  results  are  that  under 
appropriate  smoothness  assumptions  the  saddlepoint,  or  value,  V°(*)»  satisfies 
a  Hamilton-Jacobi  type  partial  differential  equation.  The  analysis  also  produces 
an  implicit  definition  of  the  optimal  strategies  in  terms  of  the  states  and  the 
components  of  the  gradient  of  V°(*).  Solution  of  the  HJE  leads  to  closed  form 
explicit  definitions  of  the  control  laws  u°  and  v°  which  induce  the  saddlepoint. 
Thus,  implementation  of  the  control  laws  requires  that  x(t)  and  y(t)  be  a\ailable 
to  each  player  at  every  t. 

The  class  of  DGWITL  which  are  studied  here  are  those  in  which  the  evader  E 

has  access  to  x(t-a)  instead  of  x(t)  at  each  time  t.  As  shown  in  [15],  this  leads 

naturally  to  consideration  of  control  laws  u(*)  and  v(')  which  are  maps  on 
n  1  n  1 

R  x  R  x  R  x  R  so  that  temporal  references  as  well  as  spatial  data  may  be 
considered.  This  also  leads  to  study  of  the  following  coupled  dynamical  systems: 

x(t-o)  =  f(x(t-o)  ,u(x(t-cr)  ,t-o,y(t)  ,t) ,  t-o)  (2a) 


^  y(t)  -  g(y (t) ,v(x(t-o) , t-o ,y (t) , t) , t) 


(2b) 
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with  x(t  -a)  *  x  ,  y(t  )  -  y  .  This  system  is  one  which  the  evader  can  think 
o  oo  o  1  o 

of  as  defining  the  evolution  of  his  observations  in  time.  The  usual  structural 
assumptions  on  u(*)  and  v(*)  are  made  here  to  provide  the  existence  of  a  unique 
solution  to  (2). 

We  then  address  the  question  of  termination.  Upon  receipt  of  data  (x,t,y,t) 

E  must  first  decide  whether  the  game  is  over,  i.e.,  could  P  have  applied  controls 
over  [t,t]  in  such  a  manner  that  the  path  emanating  from  x  puts  the  present  phase 
on  S?  This  leads  to  an  open  loop  boundary  value  control  problem  and  the  following 
definitions. 

Def.  The  preliminary  target  set,  S^,  is  the  set  of  all  (x,t,y,t)  t  Rn  x  R1  x  Rn  x  R* 

having  the  property  that  there  exists  a  piecewise  continuous  u(-):[T,t]  ■+  1.^  and 

(x,T,y,t)tS,  where  x  is  understood  to  be  the  solution  of  (la)  at  time  t  x  when 

driven  by  u  from  (x,t).  Members  of  Sp  will  be  called  potentially  terminal  phases. 

Def.  The  potential  terminal  time,  t  (x  ,t  ,y  ,t  ,u,v)  is  the  smallest  t  such 

p  oa  o  o  o 

that  the  motion  of  (1)  is  a  member  of  S^.  Figure  1  illustrates  a  potentially 
terminal  phase  and  a  preliminary  target  set.  Various  u(-)  may  be  available  for 
termination,  and  E  has  no  way  of  knowing  which,  if  any,  have  been  used  by  P. 

Def,  For  a  potentially  terminal  admissible  pair  (u,v)  we  define  the  remaining 


payoff: 


Jr(xo(j,To,yo,to,u,u,v)  =  t/  P  L^(x(a) , u(x(a) ,a,y (a+a) ,a+o) ,a)da 


L. 

+  tf  P  L1(y(o),v(x(a-a),o-o,y(a),a),a)da 


t  .  n, 

+  t/_£  L^xCa)  ,  u(a)  ,a)da  +  W(x(tp)  ,y(tp)  ,  tp) 
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Def.  The  potential  terminal  payoff  on  is  given  by: 


W(x.T,y,t)  *  min  W (x(t  ) ,y (t  )  ,t  )  + 
ueK  P  P  P 
u 


r  P  — 

T/  L^CxCa) ,u(ct) ,a)da 


Def.  The  potential  payoff,  Jp(*)»  is  defined  to  be: 


J  (x  , x  ,y  ,t  )  =  W(x(t  -a),t  -a,y(t  ),t  ) 
p  o  o  Jo'  o  p  p  J  p  *  p 


+  /  p  L, (x(a) ,u(x(a) ,a,y(a+a) ,a+a) ,a)da 

T_  i 


t- 

+  tf  P  L„(y(a)  ,v(x(a-a)  ,a-a,y(ot)  ,a)da 


The  problem  of  DGWITL  is  that  of  finding  the  potential  value,  V°(*)»  or  the 
saddlepoint  of  J  (•)  w.r.t,  u  and  v  (we  assume  that  V°(*)  exists).  With  each 
(x, t,y,t)  we  associate  V°(x,T,y,t),  and  the  results  of  [15]  show  that,  under 
the  assumption  that  V°(*)  is  continuous 1>  differentiable  in  its  argument 
in  G-Sp,  the  potential  value  satisfies  the  following  generalized  Hamilton 
Jacobi  equation: 


+  H°(x  t  y  t  V°  V°)  =  0 

aT  at  ^.T»y»t,vx,vy^  u 


where: 


H°(x,T,y,t,V°,V°)  =  H(x, t,y ,t,V°  V® ,kx  (x, T,y , t,V° , Vn) ,k  (x, T,y , t,V° ,V°» , 
jr  y  x  y  *  x  y 
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min  max  H(x,T,y,t,v,8,A  ,A )  -  max  min  H(x,T,y,t,p,B, A  ,A )  - 

u  e  *  y  b  y  *  y 

■  H(x,T,y,t,k1(x,T,y,t,Ax, Ay) ,k2(x,T,y,t,Ax,Ay) >Ax»Ay) 

and 

H(')  -  <Ax,f(x,w,T)>  +  <Ay,g(y,6,t)>  +  L^x.y.r)  +  L2(y,B,t) 

with 

p  e  A  CRq,  $  e  A  C  Rm,  A  ,A  e  Rn. 
u^“  ’  v  *  x  y 

The  generalized  Hamilton-Jacobi  equation  is  defined  on  the  playing  space  for 
P  and  E  and  over  the  T-t  "data  reference  plane";  its  boundary  condition  is 
given  by  the  potential  terminal  payoff,  W(-)  ,  on  Sp.  The  novel  feature  of  this 
equation  is  the  presence  of  t,  the  time  corresponding  to  the  delayed  state 
variable  x.  In  optimal  control  problems  and  in  DG  with  perfect  information, 
it  is  implicit  that  all  spatial  variables  are  associated  with  a  single  time,  t. 

The  procedure  for  forming  the  generalized  Hamilton-Jacobi  equation  specifies 
the  control  law  for  E  which  implements  the  data  (x,T,y,t)  in  an  optimal  fashion 
while  accounting  for  the  age  of  the  data. 

Ill  LINEAR  REGULATOR  REVISITED 

We  now  present  the  linear  regulator  as  an  example  to  illustrate  application 
of  the  theory  of  DGWITL  to  a  tractable  problem.  The  systems  are  described  by 

x  =  A  (t)x(t)  +  B  (t)u(t);  x(x  )  -  x 

p  p  O  00 

y  =  Ag(t)y(t)  +  Be(t)v(t);  y(tQ)  -  yQ, 

with  u(t)  e  AuC.Rq>  v(t)  e  A^C^'. 

The  target  set  is  given  by:  S  -  {(x,t,y,t)  :  t  -  T},  therefore,  the 

preliminary  target  set  is  simply  Sp  =  {(x,t,y,t)  :  t  =  T-o,  t=T},  where 


I 
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o  A  T  -t  .  The  terminal  time  is  t,  =  T,  and  the  potential  terminal  time  is 
*00  f 

t  (x  , t  ,y  ,t  ,u.v)  =  T,  since  the  game  has  specified  duration.  The  remaining 
p  00  o  0  o 

payoff  function  is 

Vxot’ Wt0'‘,,“’v)  "  + 


/~°IM  Ir  <o)d“  +  T^1  l“l  Ir  (a)d“  -  t/T|  iu|  !R  <a)d<’ 

op  p  o  e 


where  F  ^  0,  Rp(0  >  R0(*)  >_  0  y  ]|u 


i|2 

1  >R  (a) 


denotes  u' (a)Rp(a)u(a)  and  all 


matrices  tre  of  consistent  dimension.  The  potential  terminal  payoff  on  Sp  is  the 

solution  to  the  following  optimal  control  problem.  Let  (x,r,y,t)  e  S  ,  i.e., 

P 

t  =  T-o  and  t=T.  Then: 


W(x,I-o,y,T)  =  min  | |x(T)-y| |  +  t/T| |u 


ueK 


F  '  T-o 1  1  “ 1 ir  (a) 


da 


and  is  the  set  of  piecewise  continuous  u:  [T-o,T]  -*•  R  .  Function  space  or 
other  methods  can  be  applied  to  this  problem,  and  we  simply  provide  here  the 
result: 


W(x,T-a,y,T)  =  [$  (T,T-o)x-y ]  ’  [F-1  +  SR^S*]  *[♦  (T.T-o)x-y] 


where 


SR_1S*  A  /T  <J>  (T,a)B  (a)R_1(a)B'  (a)<f>’  (T,a)da 
p  —  T-ct  p  p  p  p  p 

and  <j>  (•)  is  the  transition  matrix  for  A  (•)•  The  optimal  terminating  control, 

P  P 

u,  is  given  by 

u(a)  =  -Rp1(a)B^(a)^’(T,a)[F"1  -f  SR^S*]"1 (T.T-a)x-y ]  ,ae  [T-o ,T] 

Thus,  u(‘)  is  E’s  determination  at  time  T  of  the  best  control  P  could  have  applied 
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V°  +  V*  +  x’A'(t)V0  +  y‘A’(t)V° 

T  t  D  X  •'e  y 


yVe,B  (t)R"1(t)B,(t)V°  +yV°’B  (t)R-1(t)B * (t)V°  «  0 
4  x  p  p  p  x  4  y  e  e  e  y 


with  boundary  condition: 


V°(x,T,y,t) 


-  [^p(T,T-o)x-y] '  [F  1+SR~1S*]~1[4ip(T,T-o)x-y] 

x=T-a 

t=T 


•  « 

X 

i 

V(T,T-o)* 

[F_1  +  SR~1S*]~1[d>  (T,T-o)  !  -I] 

X 

y 

-I 

P  P  1 

y 

where  I  denotes  the  identity  matrix  for  Rn.  We  point  out  here  that  letting 
a-K)  and  T-*t  produces  the  well  know  results  for  DG  without  time  lag.  Following 
the  procedures  for  linear  optimal  control  and  DG  we  next  assume  that 

V°(x,i,y,t)  = 

where  P(*)  is  a  2n  x  2n  symmetric  matrix. 


P(x,t) 


len 


V°(x,T,y,t)  =  2(Pl;1(T,t)x  +  P12(x,t)y) 

V°(x,x,y,t)  =  2(P12(x,t)x  +  P22(x,t)y), 
and  we  are  naturally  led  to  the  condition: 

+  P(x,t)B(x,t)P(x,t)} 


(|£(x,t)  +  -—(T,t)  +  P(x ,t)A(x, t)  +  A'(x,t)P(x,t) 
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in  G  -  S  with 
P 


A(t , t) 


Ap(,)i  0  1 

0  !A(t)J 


B(t, t) 


-B  (t)R~1(t)B* (t)  i 

~E _ E _ E _ 1. 


1  B  (t)R' 


0 

"VjB’Ct). 


and  P(t 


t)  =  P12(T,t)1 
p22(7*t)J 


Since  the  condition  holds  for  arbitrary  x  and  y,  the  following  generalized 
matrix  Riccati  equation  must  be  satisfied  over  the  x-t  plane  with  x  <  t  <  T: 


JiP  jvp 

^r(T»t)  +xr(T»t)  +  P(x,t)A(x,t)  +  A' (x,t)P(x,t) 


+  P(x,t)B(x,t)P(T,t)  «  0 


subject  to: 


P(x,t) 


x=T-o 

t=T 


<^(T,T-o) 


[F_1  +  SR~1S*]"1[<|)p(T,T-a)  i  -I], 


From  the  sufficiency  theorem  for  DGWITL  [15]  we  conclude  that  the  non-realizable 
optimal  exploitation  strategy  for  the  pursuer  is 


u°(x,x,y,t)  =  -R~  (x)B^(x)  [P^XpOx  +  P12(x,t)y], 


the  DGWITL  optimal  strategy  for  the  evader  is: 


v°(x,x ,y,t)  =  R~  (t)B^(t) [P12(x,t)x  +  P22(x,t)y], 
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and  the  potential  value  Is 


V°(x,x,y,t) 


P(t,0 


in  the  neighborhood  of  (T-a,T)  in  which  the  solution  to  (4)  is  positive  definite. 
Although  the  quadratic  payoff-linear  control  structure  of  the  solution  has  been 
preserved,  it  is  important  to  note  that  the  generalized  Riccati  equation  is  defined 
on  two  temporal  parameters,  while  the  potential  value  is  defined  on  the  spatial 
variables  and  their  associated  temporal  parameters.  This  point  is  emphasized  by 
Figure  2,  which  illustrates  the  so-called  "data  reference  plane"  for  this  problem. 
The  shaded  region  of  the  plane  below  a=0(x=t)  is  that  region  over  which  (4)  must  be 
solved.  The  line  t=T  is  the  locus  on  which  the  potential  terminal  payoff  provides 
boundary  data  for  V°(*)  over  the  shaded  region.  If  the  DG  with  zero  time  lag  has 
already  been  solved,  it  may  be  used  to  provide  boundary  data  along  o“0.  The 
behavior  of  V°  along  a  line  of  fixed  t  with  x  <  t  is  the  "aging  data  problem,"  a 
problem  of  practical  interest. 

In  general,  the  generalized  HJE  over  the  x-t  plane  must  be  converted  to  a 
"delayed  argument"  partial  differential  equation  in  t  along  the  line  x  =>  t-o.  So, 
we  must  convert  the  generalized  Riccati  equation  in  x  ar.d  t  into  a  Riccati  equation 
in  t  with  parameter  a  appearing  in  delayed  arguments  of  the  time  varying  matrices. 
That  is,  we  wish  to  generate  solutions  in  parametric  form  by  solving  the  GHJE  along 
a  fixed-o  locus  for  a  DGWITL  in  Figure  2.  For  a  fixed  o,  let 


P(t;o)  4  P(x, t) 


x*t-o 
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Then: 


f  <f,C)  -  £ <T,t) 


dr 

dt 


+ 


3P 

3t 


(T  u) 


T“t-0 


T-t-0 


It  follows  directly  that  the  solu'.ion  to  (4)  along  the  line  t  ■  t-o  with 
boundary  condition  (5)  must  se^lsfy: 


^P(t;o)  +  P(t;o)A(t-o,t)  +  A’  (t-o ,  t)P(t;o) 


subject  to: 

P(T;a) 


^(T,T-o) 


-  I 


+  P(t;a)B(t-a,t)P(t;o)  *  0 


[F"1  +  SR~1S*]“1[4  (T,T-a)|  -I] 


(6) 


(7) 


To  indicate  the  solution  along  t  **  t-a  we  write: 

V°(x,y,t;a)  = 

u°(x,y,t;o)  -  -Rp1(t-a)Bp(t-a) [P^^(t;o)x  +  P12(t;a)y] 

v° (x,y,t;a)  »  Re1(t)Bg(t)  [P12(t;a)x  +  i,12(t;a)y] 

Figure  3  illustrates  the  block  diagram  for  the  evader's  DGWITL  feedback 

control  law.  For  additional  emphasis,  we  write  x  (•)  ■  *:(•)  and  x  (•)  ■  y('). 

P  ® 

While  the  linear  feedback  structure  has  been  preserved,  the  DGWITL  control  law 
is  distinct  from  the  control  law  for  the  game  with  zero  time  lag.  The  important 
difference  is  that  the  control  law  uses  the  pursuer's  delayed  state  vector  and 


’P(t;a) 


Riccatl  gain  function  obtained  from  the  solution  to  a  Rlccatl  equation  \wi th 
delayed  arguments. 

As  an  alternate  scheme,  one  can  define 


Then: 


—  “ 

U°(‘) 

-R  ^<t-o)B'  (t-a)<(>’  (t,t-o)l  0 

p  p  P  1 

fret ;a) 

♦p(t, t-o)x(t-o) 

i 

V°(-) 

0  !  R"1(t)B'(t) 

L  e  e  J 

y(t) 

where  p(*;o)  satisfies  th;  Rlccatl  equation  In  the  control  law  block  diagram  of 
Figure  4.  This  representation  of  v°(0  exposer  the  "simple  predictor"  portion  of 

%  *\j 

the  control  law,  i.e.,  x  (t)  »  <J>  (t,t-o)x  (t-o).  The  evader  can  form  x  (t)  with 

P  P  P  P 

certainty,  since  It  determined  solely  by  P's  dynamics.  This  control  law  also 
requires  the  solution  of  a  Riccati  equation  with  delay  arguments.  It  is  clear 
from  this  representation  that  the  effect  of  introducing  the  time  delay  is  to 
create  both  a  predictor  and  a  a-dependent  Riccati  gain  in  E's  optimal  strategy. 

IV.  CONTROL  LAW  COMPARISON 

In  practice,  an  information  delay  may  not  be  detected  by  E.  E  would  then 
be  led  to  synthesize  the  DGWFI  control  law  in  ignorance  of  the  fact  that  x(t-o) 
was  being  observed  instead  of  x(t).  Or,  an  aware  E  may  simy  y  conjecture  that 
the  delay  is  of  no  consequence.  We  call  t.ne  control  law  which  uses  the  delayed 
state  in  the  DGWFI  law  a  "direct  insertion  feedback  law"(DI~'K).  Another 
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possibility  is  that  E  can  simply  predict  -  in  the  manner  already  mentioned  - 
P's  present  state  and  continue  to  use  the  Riccati  gain  of  the  DGWFI  law.  These 
two  suboptimal  control  laws  nay  be  appealing  because  they  don't  require  the  use 
of  delayed  arguments  in  the  Riccati  equation  solutions  for  their  feedback  gain 
functions.  One  might  even  expect  reasonable  performance  from  the  predictor  law 
since  it  synthesizes  a  portion  of  the  DGWITL  law.  Both  of  the  alternate  control 
laws  can  be  formed  in  Figure  4  by  setting  a=0  in  both  the  predictor  and  gain 
generator  and  in  only  the  gain  generator,  respectively.  In  applying  game  theory 
we  would  like  to  know  whether  it  is  important  to  take  into  account  i.  formation 
time  delays  in  a  competitive  situation,  or  whether  it  is  sufficient  to  use  one 
of  the  suboptimal  schemes  described  here.  Although  the  answer  to  these  questions 
depends  on  the  individual  DG,  we  provide,  by  means  of  an  example,  a  quantitative 
illustration  of  the  effect  of  information  delays,  with  the  intention  of  giving 
both  an  insight  about  the  approach  that  can  be  taken  for  the  study  of  other 
examples  and  a  demonstration  of  the  importance  cf  such  consideration  in  practical 
problems.  We  demonstrate  that  even  small  information  lags  (relative  to  the  time 
constants)  can  lead  to  serious  performance  degradation  if  the  delay  is  ignored 
and  the  DIFBK  law  is  used.  At  the  same  time,  a  simple  predictor  may  likewise 
be  inadequate.  We  also  show  that  the  DGWITL  law  effectively  compensates  for  the 
information  time  lag  and  results  in  a  marked  improvement  in  performance.  In 
addition  to  comparing  these  control  laws,  we  examine  the  effect  of  the  time  lag 
on  the  existence  of  the  solution  to  a  DG  example,  i.e.,  the  effect  of  time  lag 
on  the  occurrence  of  a  conjugate  point  in  the  delayed  argument  Riccati  equation 
solution. 

To  illustrate  the  various  point?  in  the  preceding  discussion  we  consider  the 
scalar  version  of  the  linear  regulator  results.  The  equations  cl  motion  are 
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x  =  a  x  +  b  u 
P  P  P  P 


x  -  a  x  +  b  v, 
e  e  e  e 


and  all  parameters  are  scalar  constants.  The  remaining  payoff  is  taken  to  be: 

J  O)  -  |x  (T)  -  x  (T)  | 2  +  ~  /T'a u2(a)da 
r  P  6  C  T 

p 


-  —  /Tv2(a)da  +  —  f1  u‘t(a)da, 
Ce  C  cp  T-a 


1  J  2/ 


with  a  *  T-t,  cp  >  0,  cg  >  0  and  T  fixed.  The  potential  terminal  payoff  is  the 
solution  to  the  optimal  control  problem  given  by 


W(x  (T-a) ,T-a,x  (T) ,T)  =  _min{|x  (T)  -  X£(T)|2  +  ~  fT  u2(a)da) 


c  T-a 
P 


subject  to  xp  -  apxp  +  bpu,  T-a  <  t  <  T,  x  (T-a)  given. 


The  solution  can  easily  be  shown  to  be: 


W(xp(T-a),T-a,xe(T),T) 


[e  Px_(T-a)  -  x  (T)]2 


c  b  2a  a 

x  +  ^[.  »-U 

P 


The  potential  payoff  is: 


[A0*  (T-o)  -  Xe<T)]2  j  T_„2 

J  *  - — — - +  —  /  U  (.)da 

P  5  c  t 

c  b  2a  a  P 

1  +  -S^[*  "-n 

P 


1  T  2 

~  t/V(-)da. 

e 
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By  performing  the  analysis  which  was  deacribed  in  the  preceding  section,  we  are 
able  to  show  that  the  potential  value  and  the  optimal  strategies  have  the  following 


solution: 


v°(VT*V>  ■ 


r  VT-T) 

[e  K 


x_  -  e 


a. (T-t)  , 

xe3 


G( r,t) 


.  f  ap(T-T>  ae(T~t)  a  (T-t) 

»•(«  .^s! i! - !Ip  ~  °  Xe^  e 

G(T,t) 


a  (T-t)  a  (T-t)  a  (T-t) 


V°(Xp,T,Xe,t)  - - ~ 


_cebefe  P  xn“e  x  ]  e  e 


G(t,t) 


where: 


G(t,t) 


c  b  2a  (T-t) 

1  +  ~2a^  [  e  P  -i) 
P 


Cebe  f  2ae<T-'> 

TT  (•  -U 


Alternately: 


v°(xp»xe»t;o)  - 


a  (T-t+o) 

[e 


a. (T-t) 


x  (t-o)  -  e  e  x  (t) J2 


G(t;o) 


u°(xp»xe»t;o)  - 


a  (T-t+o) 
-cbje  p 


ae(T-t)  a  (T-t+o) 

yt-g)  -  a  xe(t)]  e  P 


G(t;o) 


-  19  - 


v#(Vxe’t;0) 


a_ (T-t+o) 
e  eJ  _£■ 


-Cj b  fe  P  x  (t-o)  -  e  e  x  (t)]  e  e 


a^(T-t)  a^(T-t) 

,'t)l  e 
e 


(17) 


C(t;o) 


and 


*  cd  2a  (T-t+c) 

G(t;o)  -  1  +  [e  p  -1] 

P 


c  b  2a  (T-t) 
e  e  ,  ev  . , 

2a  [e  -11 

e 


(18) 


Figure  5  contains  the  block  diagram  for  (15),  with  the  "simple  predictor"  portion 
highlighted. 

It  is  easily  verified  that  the  conjugate  point  of  the  Riccati  gain  matrix 
corresponding  to  (15)  is  solely  determined  by  the  zero  crossing  of  G(t;o)  in 
the  neighborhood  of  T,  and  ultimately  on  the  relative  value  of  the  parameters 
in  (8),  (9)  and  (10). 

To  explore  the  effect  of  information  delay  on  the  existence  of  solutions  to 

the  linear  DGWITL ,  we  examine  (14)  and  seek  the  locus  of  conjugate  points  in  the 

data  reference  plane.  Letting  tCR(t)  denote  the  value  of  c<T  at  which  (14)  has  a 

zero  crossing  we  are  able  to  locate  the  conjugate  points  for  fixed  system  parameters. 

For  our  purposes  it  is  enough  to  present  the  results  in  Figure  6,  which  show  Tr„(t) 

vs  t  for  various  system  configurations,  with  bp  =  0.8,  cp  “  ce  ~  1-0*  ap  *  -0.2, 

a  ••  -0.4,  and  T  =  5.0.  The  area  under  the  curve  defined  by  ?-,„(•)  is  the  region 
6  CR 

A 

of  the  plane  for  which  G(*)  is  positive.  Lines  drawn  parallel  to  and  below  the 
line  T*t  correspond  to  DGWITL.  For  b  *1.35  the  line  r*t  does  not  intersect  t__, 
so  neither  the  DGWFI  or  the  DGWITL  have  conjugate  points.  For  bg  *  1.4  the  DGWFI 
has  a  conjugate  point  at  t  *  3.5  secs.  A  DGWITL  with  0<a< 0.7  also  has  a  conjugate 
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point,  and  the  time  at  which  it  occurs  is  farther  from  T  than  the  time  associated 
with  the  DGWFI  conjugate  point.  Hence,  the  information  delay  extends  the  half¬ 
interval  about  T  for  which  (18)  is  valid.  For  a  >  0.7  the  DGWITL  does  not  have  a 
conjugate  point.  Thus,  if  sufficient  information  delay  is  present  the  conjugate 
point  vanishes.  For  bg  ■  1.45  the  DGWFI  has  a  conjugate  point  at  t  »  3.8.  The 

a  a. 

DGWITL  also  has  a  finite-time  conjugate  point  at  t(o).  For  0*3,  t  *  -1.25.  These 
three  choices  of  bg  demonstrate  that  the  information  delay  effectively  lengthens 
the  solution  interval  about  T  and  may  extend  it  to  -°°,  depending  on  the  problem 
parameters.  Similar  results  can  be  obtained  for  formulations  in  which  the  pursuer 
has  an  information  delay. 

To  demonstrate  the  importance  of  properly  compensating  for  information  time 
lag  in  a  differential  game,  we  now  consider  three  control  laws  which  may  be 
implemented  in  a  DGWITL.  The  first  control  law  is  the  DGWITL  law  of  Figure  5. 

The  "direct  insertion  feedback"  law  (DIFBK)  is  synthesized  by  using  the  perfect 
information  control  law  in  spite  of  the  presence  of  delayed  data.  In  Figure  5 
this  corresponds  to  setting  o* 0  in  the  Riccati  gain  and  in  the  predictor.  By 
ignoring  the  time  delay,  E  synthesizes  this  law.  The  third  law  we  synthesize  is 
the  "predictor  feedback"  law  (PRFBK) .  This  law  approximates  the  DGWITL  law  by 
setting  a*0  in  the  Riccati  gain  while  maintaining  the  correct  a  in  the  predictor. 

In  general,  E  might  be  motivated  to  do  this  by  the  requirement  of  solving  the 
generalized  Riccati  equation  when  the  various  system  matrices  are  time  varying. 
Certainly  thia  might  be  a  reasonable  scheme  when  P(.;cr)  is  not  sensitive  to  o  over 
the  interval  of  interest. 

To  compare  these  control  laws  we  simulated  (8)  and  (9)  and  evaluated  (10)  on 
a  digital  computer,  using  Runge-Kutta  integration.  Because  the  pursuer  law  given 
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by  (16)  is  not  physically  realizable  (see  [15])  we  simulated  a  pursuer  that 
used  the  following  DGWF1  control  law: 


a  (T-t) 

0/  . *  -c  b  [e  p  x  (t)  -  e 

u  (xp,xe’t)  "  P  P  p 

G(t;0) 


ae (T-t)  a  (T-t) 

xe(t)]e  P 


For  the  purpose  of  simulating  this  control  law,  E's  path  history  over  [t^-o.t^ 

was  taken  to  be  the  autonomous  motion  of  (9)  with  terminal  boundary  condition 

given  by  y(t  ).  This  corresponds  more  closely  to  a  physical  situation  in  which 
o 

E  is  inactive  prior  to  being  alerted  that  the  conflict  has  commenced.  For  the 

purposes  of  simulation,  the  following  parameters  remained:  a^  *  -.2,  «  -.4, 

b  =  .8,  b  =  1.2,  c  *  1.0,  c  =  1.0,  T  =  5.0.  For  a  fixed  x  (t  -o)  and  x  (t  ) 
p  e  p  e  po  eo 

we  then  chose  t  ,  t  £  5.0  and  simulated  the  game  for  various  a,  with 
o  o 

0  £  o  <.  5.0.  Thus,  each  choice  of  the  pair  (tQ-o)  corresponds  to  a  different 

game.  Letting  tQ  +  T  results  in  simulations  having  a  shorter  interval  of  play 

for  E.  Since  the  physical  data,  x  (t  -a)  and  x  (t  ), remain  fixed  for  all 

p  o  e  o 

simulations,  increasing  a  effectively  changes  the  time  corresponding  to  P's 

location  at  the  fixed  x  .  This  corresponds  to  the  practical  problem  of  determining 

P 

how  the  outcome  varies  as  a  function  of  the  observation  time  associated  with 

the  fixed  spatial  data.  In  general,  control  over  starting  times  and  initial  data 

may  not  be  allowable,  so  the  important  comparison  to  be  made  is  that  which  relates 

the  payoffs  associated  with  the  three  control  laws  for  a  given  t  ,o,  x  (t  -a), 

*  j  o  p  o 

and  x  (t  )• 
e  o 

By  letting  x^^-cr)  ■  10.0  and  xg(tQ)  “  20.0  we  obtain  the  potential  value, 

V°,  directly  from  (15).  The  behavior  of  V°  is  displayed  in  figure  7.  Figures 
8-10  illustrate  the  simulated  JTL>  J^,  and  JRR,  the  remaining  payoffs  when  the 
time  lag  control  law,  the  direct  insertion  feedback  law,  and  the  predictor 


feedback  law  are  used,  respectively,  by  E.  The  graphs  show  the  behavior  of 
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these  payoffs  for  games  beginning  at  various  times  and  having  various  information 

lags.  Examination  of  the  curves  for  a  given  tQ  and  o  shows  that  the  DGWITL 

control  law  bounds  the  payoff  from  below  by  the  potential  value.  Since  P  can't 

realize  the  optimal  exploitation  control  law,  is  actually  better  than  V°.  At 

the  same  time,  the  DIFBK  and  PRFBK  laws  permit  large  negative  payoffs.  It  is 

also  of  interest  to  note  that  serious  performance  degradation  can  occur  when  the 

information  time  lag  is  ignored  (DIFBK) ,  even  for  data  lags  that  are  small 

relative  to  the  time  constants  of  the  systems,  e.g.,  for  tQ  ■  0.0  and  1.0 

(the  system  time  constants  are  2.5  secs,  and  5.0  secs.).  Figure  11  illustrates 

the  relative  performance  of  the  control  laws  for  t  “0.0  by  showing  the  loss 

o 

in  payoff  incurred  by  DIFBK  (nDI)  and  PRFBK  (npR)  normalized  by  the  maximum 
loss  incurred  by  DIFBK  for  0  <,  o  £  5.0. 

To  interpret  the  numerical  data  in  more  detail  it  should  be  noted  that, 

because  the  plants  are  stable  and  the  initial  cr-rdinates  were  both  taken  to  be 

positive,  the  natural  motion  of  (8)  over  [to-a,tQ]  works  to  P's  disadvantage  - 

in  the  sense  that  if  P  was  inactive  over  [to-a,tQ]  the  actual  separation  at 

time  tQ  would  be  greater  than  |xp(tQ-a)  -  xg(to)  |.  In  Fig.  7  the  potential 

value  is  always  bounded  from  below  by  zero,  since  E  can  choose  to  do  nothing 

in  (9).  For  t  *  5.0  the  solution  corresponds  to  the  potential  terminal 
o 

payoff. 

Figure  12  shows  the  behavior  of  J^,  JQI  and  JpR  for  the  same  spatial  data 

but  with  t  “  -5.0.  For  this  game  of  longer  duration  (10  secs.)  non-negligible 
o 

payoff  degradation  accompanies  DIFBK  and  PRFBK  control  even  for  relatively 
small  information  delays.  Fig.  13  shows  the  P  and  E  trajectories  resulting  from 
DGWITL  and  DIFBK  control  by  E.  Although  the  terminal  miss  for  DIFBK  is  better, 
E's  net  performance  is  worse  because  the  DIFBK  law  erroneously  uses  excessive 
energy  in  the  terminal  phase  of  play.  Figure  14  shows  the  remaining  payoffs 
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for  a  game  of  5.0  secs,  duration  with  different  spatial  coordinates.  Here 

the  detrimental  use  of  the  DIFBK  law  is  very  apparent.  Figure  15  shows 

the  payoffs  for  the  same  spatial  data  but  for  play  of  10  secs,  duration. 

Again,  it  is  clear  that  ignoring  the  information  delay  can  lead  to  serious 

performance  degradation.  At  the  same  time,  even  the  predictor  control  law 

leads  to  what  might  be  considered  to  be  unacceptable  performance.  On  the  other 

hand,  the  DGWITL  control  law  appears  to  lead  to  performance  that  is  much 

less  sensitive  to  the  information  delay. 

To  display  the  effect  of  the  relative  location  of  P  and  E  w.r.t.  the 

origin,  we  let  x  (t  -a)  =  20.0  and  x  (t  )  *  10.0.  Figure  16  shows  the  plot 
p  o  e  o 

of  the  potential  value.  For  ;iven  tQ,  V°(0)  decreases  w.r.t.  0  for  the 

range  of  °  shown,  except  for  tQ  =  5.0,  where  a  slight  increase  occurs  near 

a  =  5.0.  The  main  effect  causing  this  decrease  of  potential  value  as  the  data 

ages  is  the  natural  component  of  P’s  motion  over  [tQ-a,to].  This  tends  to  place 

the  actual  x  (t  )  closer  to  x  (t  )  with  no  effort  on  behalf  of  P.  For  o  beyond 
p  o  e  o 

3.5  secs.,  however,  this  effect  works  to  E's  advantage. 

Figures  17,  18  and  19  show  J^,  JDI  and  JpR  for  the  various  tQ  and  o. 

The  superiority  of  the  DGWITL  control  law  is  evident.  Figure  20  shows  these 

payoffs  for  t  =  -5.0  (i.e.,  a  10  sec.  game).  ' 'ain,  the  payoff  degradation 

due  to  failure  to  compensate  properly  for  the  information  delay  is  evident. 

As  an  alternate  scheme,  we  simulated  the  situation  in  which  the  pursuer 

was  inactive  over  [t  -a, t  ],  thus  the  motion  of  both  P  and  E  was  autonomous 

o  o 

over  this  interval.  Figures  21,  22  and  23  contain  plots  of  xp(to-o)  *  10.0 
and  x  (t  )  =  20.0.  Here  we  see  that  for  large  a  and  a  short  interval  of  play 

6  O 

(e.g.,  t  -  4.5  secs.)  the  DIFBK  law  does  slightly  better  than  the  DGWITL  law. 
Since  P  is  not  synthesizing  the  exploitation  control  law,  this  is  possible. 
However,  the  DIFBK  law  does  not  provide  E  with  the  payoff-bounding  property. 
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Also,  for  small  o  the  superiority  of  the  DGWITL  law  is  retained.  In  comparing 

Figs.  21-23  with  Figs.  6-10,  we  expect  the  greatest  difference  In  performance 

to  occur  for  larger  o,  where  the  effect  of  the  different  initialization  scheme 

is  more  pronounced.  Fig.  24  illustrates  the  performance  due  to  the  three 

control  laws  for  t  -  0.0  and  x  (t  -a)  ■  10.0,  x  (t  )  •  20.0.  Figs.  25-27 
o  p  o  e  o 

are  for  spatial  data  of  x  (t  -a)  ■  20.0  and  x  (t  )  *  10.0.  Fig.  28  contains 

P  O  6  0 

JDj(o)  an<*  **PR^  ^or  t*'e  same  data  and  tQ  ■  0.0. 


V.  CONCLUSIONS. 

It  is  clear  from  the  few  basic  examples  presented  here  that  failure  to 
include  a  consideration  of  information  delay  in  the  synthesis  of  a  control 
law  can  result  in  serious  performance  degradation,  even  for  relatively  small 
amounts  of  time  delay.  We  expect  that  similar  phenomena  can  occur  in  more 
complex  and  realistic  problems.  Our  work  in  this  paper  serves  to  demonstrate, 
for  the  first  time,  that  detection  of  information  delays  and  determination 
of  their  significance  by  means  of  the  DGWITL  theory  should  be  part  of  the 
overall  synthesis  procedure  in  situations  of  dynamic  conflict. 

From  an  engineering  viewpoint,  the  DGWITL  theory  provides  the  framework 
within  which  the  system  designer  can  assess  the  payoff  degradation  due  to 
information  delay  and  establish  acceptable  amounts  of  delay.  The  control  law 
synthesized  by  the  DGWITL  theory  can  also  be  used  as  a  means  for  comparing 
the  performance  of  suboptimal  control  laws,  such  as  the  predictor  law  used 
in  the  examples.  In  general  it  is  necessary  to  perform  the  assessment  over 
the  regime  of  spatial  data  for  the  problem.  Some  spatial  regimes  may  admit 
the  use  of  a  suboptimal  law,  while  others  may  require  the  DGWITL  law  in  order 
to  maintain  system  performance.  Not  only  does  the  DGWITL  theory  allow  the 
system  designer  to  assess  the  payoff  degradation;  it  also  provides  the  designer 
with  an  important  engineering  option.  If  the  DGWITL  law  can  maintain  acceptable 
performance  over  a  spatial  regime,  then  the  designer  has  the  option  of  deliberately 
introducing  information  delay  so  that  a  single  slower  computer  -  data  acquisition 
system  may  be  used,  or  a  sufficiently  fast  computer  -  data  acquisition  system 
may  be  used  to  simultaneously  control  several  systems,  each  of  which  employs 
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a  DGWITL  controller  .  These  engineering  tradeoffs  underscore  the  relevance 
and  importance  of  the  theory  of  differential  games  with  information  delay. 


*  For  a  hypothetical  example,  from  Fig.  20  it  is  clear  that  the  DGWITL  law  would 
enable  use  of  a  computer  -  data  acquisition  system  which  generated  a  one  second 
information  delay;  alternately,  more  than  one  system  may  possibly  be  controllable 
by  a  faster  computer  which  presented  each  system  with  the  same  one  second  delay. 
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Fig.  3.  F.vader  DGWTTL  Control  Law. 


-  31  - 


f - i 

}*p(t.t-o)| 


Fig.  4.  Predictor  Pecomoosition  of  1X7.VITL  Control  Law. 
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